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METHOD FOR DELIVERING INFORMATION 
BASED ON RELATIVE SPATIAL POSITION 

BACKGROUND OF THE INVENTION 



1. Field of the Invention : 

The present invention relates to a method of retrieving product 
information, and more particularly to retrieving product information via a 
10 spatially-sensitive system including a portable display device. 



2. Discussion of the Prior Art : 

While retail stores offer the convenience of nearly instantaneous delivery, 
if a customer can not find what he or she is looking for, a sale may be lost 

15 Determining whether a store carries a particular item (or class of items) and 

getting directions for how to find the item in the store may improve the likelihood 
of making a sale. Moreover, seeing an item can often tip the balance and lead to 
additional impulse sales. 

Even if the customer can locate the item or category of a desirable 

20 product, the customer may be unsure whether to make a purchase due to cost, 
suitability for a particular application, confusion generated by a plethora of 
choices, etc. Typically, the role of a sales associate is to help the customer make 
an informed purchase and nudge the customer into commitment if the customer 
is wavering. However, hiring knowledgeable retail employees can be difficult. 
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Many devices exhibit forms of spatial-sensitivity, for instance, global 
positioning system (GPS) devices determine a user's location to within a few 
yards. Other examples include interfaces to virtual reality environments which 
determine and display a virtual position in a computer generated environment, 
5 metrology devices for creating computer models from physical artifacts, tracking 
devices for monitoring vehicles (e.g., Lojack®), and various traffic monitoring 
systems (both pedestrian and vehicular). These systems are used merely to 
determine a global position and lack the capability to relate multimedia 
information to a consumer. Further, a number of these systems may not work 

10 indoors due to weak communications channels. 

Systems such as cell phones, personal digital assistants (PDAs), and 
laptop computers have the ability to perform more complex tasks. However, 
these devices typically have no spatial sensitivity other than for establishing a 
connection to resources such as a specific computer local area network (LAN) or 

15 a telephone calling area (or specific cell tower). These systems do not provide 
precise positional information and may lack multimedia capabilities. 

Fitzmaurice's Chameleon device is an example of a system which 
provides information, however, it is used primarily to provide a graphical 
information overlay on a view of the real world, or as a window through which 

20 spatially registered images are viewed. The system is described in "Situated 
Information Spaces and Spatially Aware Palmtop Computers", George W. 
Fitzmaurice, Communications of the ACM . Vol. 36, No.7, pp.39-49, July 1993. 
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Therefore, a need exists for a method of providing a customer personal 
access to multimedia information about product in real time. 

SUMMARY OF THE INVENTION 

5 According to an embodiment of the present invention, a method for 

providing product information comprises establishing a local reference frame 
which defines a space including a product identifier and a portable display 
device, receiving a request for product information corresponding to the product 
identifier from the portable display device, determining a position of the portable 

10 display device in relation to the local reference frame, and providing the product 
information via the portable display device according to the position of the 
portable display device. 

The method includes receiving an order for a product corresponding to the 
product identifier from the portable display device. The method includes 

15 receiving a bid for a product corresponding to the product identifier using the 
portable display device. The method comprises the step of adding a product 
corresponding to the product identifier to a user shopping list using the portable 
display device. The method includes providing a menu for distinguishing a 
plurality of products in the space via the portable display device. The method 

20 further includes the step of redirecting a user toward an alternative product using 
the portable display device. 
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The method further comprises determining the orientation of the portable 
display device, and selecting between the object and another proximate object 
on the basis of the orientation of the portable display device. 

The local reference frame is established using an active beacon. The 
5 method includes determining a position of the portable display device by 

comparing one of signal strengths of at least two beacons, a signal transmission 
time from each of at least two beacons, and an angle between at least two 
beacons. 

The local reference frame is established using passive environmental 
10 markings. The method comprises determining a position of the portable display 
device relative to at least one environmental marking. The method includes the 
step of determining a position of the portable display device relative to an angle 
between at least two environmental markings. 

The method retrieves the product information from a database stored in 
15 the portable display device. 

The local reference frame is established relative to the portable display 
device and moves with the portable display device. 

According to an embodiment of the present invention, a system for 
providing a user access to information comprises a portable display device within 
20 a local reference frame including an object, a plurality of positional sensors, a 

correlation means for determining the object according to a known position of the 
object within the local reference frame, and the position of the portable display 
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device, and a database for providing, via the portable display device, information 
corresponding to the object. 

The system further comprises a plurality of active beacons defining the 
local reference frame, wherein the positional sensors are part of the portable 
5 display device. 

The correlation means determines the position of the portable display 
device based on a signal strength of at least one active beacon, wherein the 
signal strength is determined by the positional sensors. The correlation means 
determines the position of the portable display device is based on a signal 
10 transmission times from each of at least two active beacons. 

The system comprises a wireless communication link between the 
portable display device and a database of product information. 

According to an embodiment of the present invention, a program storage 
device is provided readable by machine, tangibly embodying a program of 
15 instructions executable by the machine to perform method steps for providing 
product information, the method includes establishing a local reference frame 
which defines a space including a product identifier and a portable display 
device, receiving a request for product information corresponding to the product 
identifier from the portable display device, determining a position of the portable 
20 display device in relation to the local reference frame, and providing the product 
information via the portable display device according to the position of the 
portable display device. 



YOR9-2001-0313US1 (8728-508) 



5 



BRIEF DESCRIPTION OF THE FIGURES 

Preferred embodiments of the present invention will be described below in 
more detail, with reference to the accompanying drawings: 

Fig. 1 is a diagram of an beacon-instrumented store according to an 
5 embodiment of the present invention; 

Fig. 2 is an example of the external architecture of a single-sensor 
portable display device according to an embodiment of the present invention; 

Fig. 3 is an example of the external architecture of a triple-sensor portable 
display device exterior according to an embodiment of the present invention; 
10 Fig. 4 is an illustrative block diagram of a PDA's internal architecture 

according to an embodiment of the present invention; 

Fig. 5 is an illustrative block diagram of a PDA's logic according to an 
embodiment of the present invention; 

Fig. 6 is a flowchart showing the business process according to an 
15 embodiment of the present invention; 

Fig. 7 is an example of data being displayed by a portable display device 
according to an embodiment of the present invention; 

Fig. 8 shows an example of a time-of-flight-based location determining 
configuration according to an embodiment of the present invention; 
20 Fig. 9 shows an example of a image analysis-based location determining 

configuration according to an embodiment of the present invention; 

Fig. 10 shows an example of a single customer-carried reference pattern 
according to an embodiment of the present invention; and 
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Fig. 11 shows an example of a customer with multiple reference emitters 
according to an embodiment of the present invention. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

5 According to an embodiment of the present invention a local coordinate 

system is established within an area such as a store or showroom. A customer 
within the area are provided with a spatially sensitive portable display device, for 
example a personal digital assistant (PDA). The portable display device 
determines its position within the local coordinate system. Depending on the 

10 user's position and the orientation of the portable display device, product 

information relevant to the position or spatial location of the portable display 
device can presented to the customer. For example, which products proximate to 
the personal display device, for example, within a children's clothing department 
of a store, are on sale. Such a system may be provided as a service to one or 

15 more businesses within the area. The present invention may be implemented in 
many contexts, for example, indoors on a sales floor, outdoors at an automobile 
dealership, and in non-commercial spaces such as a museum. 

Referring to Fig. 1, a sales floor 1010 is shown, including a customer 1040 
carrying a portable display device 1050. The sales floor includes active beacons 

20 2010-2060 and display cases 2070, 2080, and 2100. The active beacons can 
include, inter alia, radio frequency, ultrasonic, and infrared beacons. A store 
database server 2090 is provided including information related to goods and/or 
services provided by a business or vendors on the sales floor. The customer 
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1040 can access the database server 2090 via the portable display device and a 
wireless link (e.g., including, but not limited to those supported by IEEE 
standards 802.11a and 802.11b, and developing standard 802.11g). Platforms 
able to provide the services for the store database server 2090 include, but are 
5 not limited to, IBM's DB/2 database server running on one of IBM's 390 or 

RS/6000 computer systems. Other systems would be obvious to one skilled in 
the art. 

Alternatively, the user may have already downloaded the complete store 

database into his portable display device. If this is the case, and the device is 
10 able to determine its position without help from an external system, then there is 

no need for a wireless link. The determined position can be used to look up 

information already resident in the display device. 

Figs. 2 and 3 depict examples of the external architecture of portable 

display devices according to the present invention. Fig. 2 shows an example of a 
15 portable display device 1050 equipped with a display 2010, a trigger button 2030, 

and a sensor 2020. Fig. 3 shows a portable display device 1050 equipped with a 

display 3010, a trigger button 3050, and three sensors 3020, 3030, and 3040. 

The additional sensors can be used to obtain the orientation of the display 

device, or to make the position determination process more robust. Those with 
20 regular skill in the art will appreciate that other configurations are possible in view 

of the present invention. 

According to an embodiment of the present invention, a touch sensitive 

screen is provided, for example as used by Palm™ handheld computers. Further, 
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a keypad and/or barcode scanner may be provided for data input. Such 
interfaces would allow customers to provide input in addition to the triggering 
impulse provided by the triggering buttons 2030 and 3050. This input can be 
used to request additional types of information, to complete the purchase of an 

5 item, or to request directions to some related item. 

Fig. 4 shows an example of the internal architecture of the portable display 
device 1050 applicable to the present invention. The portable display device 
includes, inter alia: a central processing unit (CPU ) 4000; a storage means 4010 
(e.g., programmable nonvolatile RAM); a wireless communications interface 

10 (e.g., including, but not limited to those supported by IEEE standards 802.1 1a 

and 802.11b, and developing standard 802.1 1g) 4020 through which the portable 
display device can communicate with other network devices (e.g., the store's 
product database); a Signal Processor 4030 for interacting with the sensors; and 
a memory 4040. The memory 4040 includes, as executable binary code, inter 

15 alia: a Data Receive Handler 4050; a Data Display Handler 4060; a Location 
Determination Handler 4070; and a Data Request Handler 4080. 

Fig. 5 depicts an example of the portable display device's logic. In block 
5000, the portable display device waits for input and, when received, checks the 
type of input in block 5010. The portable display device first determines whether 

20 the input was a trigger request, signaled by the user pressing the trigger button. 
Upon determining that an input trigger request has arrived, the portable display 
device invokes its Location Determination Handler 5020 to determine its current 
position within the area (e.g., the store). Next, the portable display device 
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invokes the Data Request Handler 5030 to request data relevant to its current 
location from the Store Database Server; control then continues at block 5000. 

If the input was not a trigger request, the portable display device checks 
if data is being returned 5040 by the Store Database Server 2090. This typically 
5 happens in response to a previous user request. However, data might also be 
sent spontaneously based on the user's current position either as a navigational 
aid, or as a location-specific advertisement feed. Upon detecting the arrival of 
data from the Store Database Server, the portable display device invokes its 
Data Receive Hander 5060 to pull in all data being sent to it The portable display 

10 device then invokes its Data Display Handler 5070 in order to the present the 

received data on its display. Finally control passes back to block 5000. If no such 
data is being received, a miscellaneous handler is instead invoked 5050, after 
which control continues directly at block 5000. 

Fig. 6 indicates the overall business process of the present invention. In 

15 block 6000, the customer enters the store and is given a portable display device. 
Alternatively, the user may already own an appropriate device which he has 
brought with him. In block 6010, after finding an item of interest, the customer 
positions the portable display device near the item or points it toward the item. In 
block 6020, the portable display device then determines its location in the store, 

20 and in block 6030 sends a request to the store's database asking for information 
relevant to the portable display device's current location. Note that the portable 
display device can be equipped with a button that would allow the customer to 
indicate when the portable display device should send such requests. In block 
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6040, the store database retrieves all information relative to the specified 
location, and then sends it back to the portable display device in block 6050. In 
block 6060, the portable display device receives the relevant information from the 
store database and then displays it to the customer. 
5 The product information can include, inter alia, sales information, 

promotional information and comparison information. By way of an illustrative 
example, and not by limitation, the portable display can automatically provide a 
customer in the printer section of a computer superstore with product reviews of 
the model he or she is proximate to (e.g., standing in front of). Other types of 

10 spatially-dependent data that might be presented include, inter alia, comparisons 
of similar items in the local area, price and inventory information about one or 
more models, or automated sales pitches for items on sale. Referring to Fig. 7, 
the portable display device 1050 can also display cross-referencing data, for 
example, the fact that the size of the selected pair of pants is "medium" and that 

15 this is equivalent to a 34 inch waist 7050. 

In conjunction with additional user input, the device can also provide 
navigation directions from the current location to the vicinity of a selected item, 
allow the entering of a bid for a nearby item, request remote notification 
concerning the availability of an item, suggest a suitable substitute product, or 

20 place an order for specific merchandise. 

There are various methods for determining the position of the portable 
display device in free-space. For example, several acoustic emitters and 
detectors (e.g., ultrasonic) can determine time-of-f light measurements along with 
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a partially known emitter/detector geometry to establish the relative position and 
orientation of the portable display device. A similar method proposes the use of 
relative phase shift measurements of an acoustic or electromagnetic signal to 
estimate path lengths between various emitters and detectors. Such systems are 

5 described in Section 6.1 of "Sensors for Mobile Robots: Theory and 
Applications", H. R. Everett, A.K. Peters Inc., Wellesley MA, 1995. 

Fig. 8 shows an example of a time-of-flight transponder-receiver 
configuration. Transponder 8800 includes an Infrared (IR) emitter 8200 and two 
ultrasonic (US) emitters 8100. At the same instant the transponder 8800 

10 generates a burst of IR light 8600 and one or more pulses of US sound 8500. 

The pulse from the two emitters 8100 can be frequency coded so that their arrival 
times can be disambiguated. Alternatively, for every IR pulse first one then the 
other of the US emitters can be triggered. The portable device 1050 first 
receives the infrared pulse 8600 via its IR receiver 8400, then awaits the arrival 

15 of the ultrasonic pulses 8500 at its US receivers 8300. This provides 6 arrival 
times which correspond to the physical path lengths between the US receivers 
8300 and the US emitters 8100. Such a system of length measurements can be 
solved to give three dimensions of spatial position and three rotation angles of 
the device 1050, These positions and rotations are relative to the local reference 

20 frame established by the transponder assembly 8800. Alternatively, the portable 
display device might emit an IR pulse which then triggers the transponder to 
generate its US pulses. This also yields a number of path lengths (signal 
transmission times) which can be combined to determine the device's position. 
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If only the position of the device is desired, then fewer receivers can be 
used. However, determining the device's orientation also allows the user to make 
finer positional discriminations by pointing the device. The location determination 
handler 4070 can take the (x, y, z) position of the device as a base point and 
5 then project a ray along the pointing direction as determined from the orientation 
of the device. The first object position that this ray intersects would then be 
chosen as the object selected by the user. This is a particularly useful mode 
when the object is out of convenient reach, such as being high up on a shelf. 
Another possibility involves measuring the angles to a number of 

10 reference points and computing the device position based on this. The azimuth 
and elevation of several infrared LED beacons can be determined using, for 
example, a two-dimensional (2D) imaging system, a linear array sensor, or a 
mechanically scanned detector. Fig. 9 shows an example of such a 2D imaging 
system. Here there are three IR emitters 9200 that blink in a particular pattern, 

15 such as a clockwise rotation where only one is on at a time. The emitters are 

imaged through lens 9300 onto a position sensing light detector 9400, such as a 
quadrant photodetector (PSD) or a CCD (charge-coupled device) camera. The 
lens and detector are part of portable display device 1050. Each imaged point 
9500 then yields an x and y image offset from the center of photodetector 9400. 

20 Combined with the focal length of lens 9300, each of these (x, y) measurements 
can be translated into a pair of angles corresponding to the direction toward the 
particular emitter 9200. If the true physical spacing between all the emitters 9200 
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is known, then the three sets of computed angles can be used to solve for the 
pose (i.e., position & orientation) of light detector 9400 and hence device 1050. 

One with regular skill in the art will also see that the whole apparatus can 
be turned around. That is, the emitters can be part of portable display device 
5 1 050 while the light detector 9400 and lens 9300 are part of the store 

infrastructure. In this case the position determination would most likely be done 
by store database server 2090 (or some equivalent computer which is part of the 
store) instead of by portable display device 1050. 

According to an embodiment of the present invention, an approximate 

10 position can also be established based on proximity to a particular reference. For 
example, the portable display device can determine its position according to 
which micro-cell a radio was operating in, the angular sector of one or more 
micro-cells, the code(s) associated with the strongest responding nearby 
beacon(s) 2010 - 2060, the reading of a barcode affixed to a pillar within the 

15 building, or even the reading of the barcode on a shelf or affixed directly to a 

product. According to the preferred embodiment, beacons 2010-2060 can be RF 
(radio frequency) emitters. The portable display device would then use sensor 
2020 and Signal Processor 4030 to determine the signal strengths of the RF 
beacon(s). The strength of such a signal is related by the inverse square law to 

20 the distance between the beacon and the portable display device. Thus, the 

strongest beacon among a group of beacons can be determined to be the most 
proximate beacon. If signals from two or more beacons were received, a position 
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somewhere in between them can be interpolated based on the known signal 
propagation characteristics. 

One with regular skill in the art will appreciate that the portable display 
device can be outfitted to serve as a reference coordinate frame, either solely or 
5 in conjunction with the store's reference frame. Examples of this are shown in 
Figs. 1 0 and 1 1 . The portable display device can determine the customer's 
location relative to itself. Given this additional relative positioning information, the 
portable display device 1050 can allow the user to choose which aspect of a 
given selected item is displayed. For example, the customer might point her 

10 portable display device at a particular dress. By holding the portable display 

device farther away from herself, she can have the device display suggestions of 
shoes that will match, while by holding the device closer, she can have it show 
size cross-referencing information. 

Fig. 10 shows an example where a passive marking 10100 is placed on a 

15 portion of the user (his glasses) to serve as a reference point. A camera on the 
portable display device can be used to measure the size, rotation, and skew of 
this marking to determine the position and rotation of the device relative to the 
user. This is equivalent to detecting several distinguished points within the 
marking and then determining the angles to these points (as was described in the 

20 discussion of Fig. 9). Alternatively, intrinsic features of the user such as the 
centers of his eyes and the tip of his nose can be used as reference points 
without requiring the addition of an explicit marking. A camera can once again be 
used to detect these features and measure the attributes of their projected 
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configuration. If the true size of the user's face is known, this system of 
measurements can again be inverted to yield the pose of the portable display 
device. A similar system using either intentional passive markings or intrinsic 
environmental features can also be used to establish the local reference frame 
5 for the whole store. 

Fig. 1 1 shows another example where the customer dons a hat equipped 
with three active emitters, 1 1 1 00-1 1 300. These emitters can be tracked by the 
store system using a number of ceiling mounted cameras to determine the 
position of the customer within the store. Different hats might blink at different 

10 rates or emit different pulse trains so that individual customers can be 

distinguished. However, these emitters can be tracked independently by the 
portable display device to determine the device's position and orientation locally 
with respect to the user. 

Note that either the device's sensors or the user's emitters can form the 

15 basis of the local reference frame. Determining the device's position relative to 
the emitters is equivalent to determining the emitter's position relative to the 
device. 

Having described embodiments of a method for retrieving data Using a 
spatially sensitive personal data assistant, it is noted that in light of the above 
20 teachings modifications and variations can be made by persons skilled in the art. 
It is therefore to be understood that changes may be made in the particular 
embodiments of the invention disclosed which are within the scope and spirit of 
the invention as defined by the appended claims. Having thus described the 
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invention with the details and particularity required by the patent laws, what is 
claimed and desired protected by Letters Patent is set forth in the appended 
claims. 
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